Skip to content

feat(runtime): add native TensorRT deployment for SenseVoice - #3463

Merged
LauraGPT merged 1 commit into
mainfrom
codex/tensorrt-deployment-3453-20260804
Aug 3, 2026
Merged

feat(runtime): add native TensorRT deployment for SenseVoice#3463
LauraGPT merged 1 commit into
mainfrom
codex/tensorrt-deployment-3453-20260804

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a native TensorRT builder for the SenseVoice encoder with ONNX contract validation, dynamic optimization profiles, FP16/FP32 selection, and atomic plan output
  • reject dynamically quantized ONNX graphs before parsing, with the exact quantize=False recovery command
  • add a Triton tensorrt_plan config template while preserving the existing ONNX Runtime default
  • document export, build, profile sizing, plan portability, and verified H100 results

Verification

  • red-first TensorRT contract tests: 10/10 passed
  • merged-main focused suite (builder contracts plus related docs/runtime checks): 31/31 passed
  • Black 24.4.0 and git diff --check: passed
  • official FP32 ONNX: checker passed; TensorRT 10.0.1 parser returned 0 errors
  • H100 FP16 build: 527,504,916-byte plan in 113.881 s, profile batch 1-16 / frames 1-4096
  • runtime boundaries: (1, 1), (8, 512), and (1, 4096) produced finite outputs
  • random 30/64-frame parity: 100% CTC top-1 agreement with PyTorch
  • bundled Chinese audio: 100% frame top-1 agreement and exact transcript 开饭时间早上九点至下午五点

The generated plan and model weights are intentionally not committed because TensorRT plans are tied to the target GPU architecture and TensorRT version.

Closes #3453

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@LauraGPT
LauraGPT merged commit 6408aaa into main Aug 3, 2026
@LauraGPT
LauraGPT deleted the codex/tensorrt-deployment-3453-20260804 branch August 3, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

有onnx转tensorRT的代码吗?

1 participant